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Abstract 


Since 2014, the Wake County Public School System has implemented single subject acceleration (SSA) 
as a way to provide students with access to advanced mathematics courses. This report includes three 
main findings related to the implementation and impact of SSA. First, a disproportionally large 
percentage of male, Asian and academically/intellectually gifted students were nominated, qualified 
and accelerated compared with their female, Black and Hispanic/Latino counterparts. Second, roughly 
two-thirds of students who qualified for SSA in mathematics actually proceeded to take the accelerated 
course. Third, near the qualifying cutoff score, accelerated students performed similarly to their non- 
accelerated counterparts, suggesting that SSA had no significant achievement effects—positive or 
negative—for students who were accelerated. We recommend that staff expand the visibility of SSA in 
order to inform more diverse populations, identify potential causes of non-acceleration among 
qualifiers, explore options for assessing content-level mastery, and maintain the 80% qualifying CASE 
score for SSA mathematics. 
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SUMMARY 


While public education in the United States has typically focused on students moving 
from grade to grade based on their age, some academically advanced students can also skip 
grade levels and content through acceleration practices. In the United States, such practices can 
be traced as far back as the late 19th century, when public schools in St. Louis experimented 
with flexible promotion policies. Half a century later, psychologist Sidney Pressey summarized 
more than a dozen different accelerative practices (Colangelo, Assouline, & Gross, 2004; 
Pressey, 1949). Some of the more well-known of these include content-based forms such as 
single-subject acceleration (SSA), curriculum compacting, and Advanced Placement. Grade- 
based accelerative practices include grade skipping, early graduation, and early admission to 
kindergarten. Despite the long history of subject-based acceleration, the extent to which SSA, in 
particular, has been effective at impacting academic achievement remains unclear. This study is 
among the first to examine the causal impact of SSA in a school district setting. 


This evaluation reports the impact of SSA in mathematics on student achievement 
outcomes in the Wake County Public School System (WCPSS).* SSA was launched in 2012-13 to 
allow students with advanced academic skills to skip a year of content in either mathematics or 
reading. Starting in 2013-14, elementary and middle school students could qualify to enroll in 
an advanced mathematics course if they score at least 80% on an above-grade-level assessment 
in spring prior to acceleration.” For example, if a student finishing 3rd grade wants to skip 4th 
grade mathematics as a 4th grader, she needs to score 80% or above on the qualifying 
assessment. If she meets this cutoff score, she may skip 4th grade mathematics and take 5th 
grade mathematics while enrolled in the 4th grade. The existence of this qualifying cutoff score 
allows for the use of a technique called regression discontinuity design (RDD) in order to 
compare differences in outcomes for students below and above the 80% qualifying cutoff. RDD 
is considered a quasi-experimental method that allows us to provide causal impacts of program 
effectiveness (Table 1). A positive and significant discontinuity in achievement between 
accelerated and non-accelerated students around the eligibility threshold would confirm what 
many proponents of acceleration already believe through anecdotal evidence: that acceleration 
can only benefit advanced students. In the event that there is no significant impact, we would 
suggest that the policy at least does no harm. A negative finding would raise questions about 
the tradeoff between exposure to advanced mathematics content and achievement on 
standardized tests. 


1 An SSA Status Report is available on the district Intranet by request (Lenard & Townsend, 2016). 
* This qualifying assessment is a version of the Collaborative Assessment Solutions for Educators (CASE) test, 
developed by TE21, Inc. 


Table 1 

Nature of the Data Provided and Valid Uses 

© Experimental We can conclude that the program or policy caused changes in outcomes 
because the research design used random assignment. 


VI Quasi-Experimental We can reasonably conclude that the program or policy caused changes 
in outcomes because an appropriate comparison strategy was used 
© Descriptive These designs provide outcome data for the program or policy, but 
QO Quantitative differences cannot be attributed directly to it due to lack of a 
QO Qualitative comparative control group. 


Sources: List, Sadoff, & Wagner (2011) and What Works Clearinghouse (Clearinghouse, 2014). 


In spring 2013, nearly 3,000 WCPSS students in grades K-7 were nominated by parents, 
teachers, or principals to take the SSA qualifying test. This number declined to roughly 1,800 in 
2014 and 1,400 in 2016. Due to data quality issues and changes in qualifying criteria in 2013, in 
this study we report on outcomes for students nominated in 2014 and 2015 only. For students 
nominated in 2014 and 2015, acceleration refers to enrollment in a mathematics course that is 
one grade level beyond a student’s current enrollment grade. An example of this timeline 
appears in Table 2. 


Table 2 
Example Timeline for an Accelerated Student 
Spring 2015 2015-16 School Year Spring 2016 

Context Student A is nominated Student A, now in Student A, after 

in spring of grade 3 to grade 4, qualifies for completing grade 5 

skip grade 4 math and => SSA and enrolls in the = math while in grade 

take grade 5 math accelerated course, 4, takes the grade 4 
while still in grade 4 which is grade 5 math End-of-Grade test 

Grade-Level Grade 3 Grade 4 Grade 4 
Content-Level Grade 3 math Grade 5 math Grade 5 math 
EOG test Grade 3 = Grade 4 


Results 


The Pathway of Change (Figure 1) summarizes outcomes of interest identified by 
evaluation and program staff. Prior work on acceleration in WCPSS showed that students 
viewed acceleration more favorably than teachers or principals; nominees had higher levels of 
engagement than non-nominees; and accelerated students and their nominated but non- 
qualified counterparts were similar on most measures of engagement (Lenard and Townsend, 
2016). This study explores (1) the demographic composition of nominees, qualifiers and 


accelerators, (2) the proportion of qualifiers who proceed to accelerate and (3) the academic 
performance of accelerated students. The results show that: 


e Male, Asian and academically or intellectually gifted (AIG) students were much more 
likely to be nominated for SSA, qualify and be accelerated compared with their female, 
Black and Hispanic/Latino counterparts. 

e Roughly two-thirds of students who qualified for SSA in mathematics proceeded to take 
the accelerated course. 

e Near the qualifying cutoff score of 80%, accelerated students performed similarly to 
their non-accelerated counterparts on the summative grade-level, state mathematics 
achievement test, suggesting that SSA functioned as a “do-no-harm’” initiative for 
students taking advanced coursework. 


Figure 1 
Pathway of Change 


Effort: Single Subject Acceleration 
Need: For the 2014-15 school year, district administrators adopted a revised criteria for elementary and middle 


school students attempting to qualify for acceleration in either reading or mathematics. The stated need is to 
provide advanced students with the opportunity to skip content if they can demonstrate mastery. 
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BACKGROUND 


Advocates of accelerative practices are motivated by the belief that every child is unique 
and, as such, can be matched to specific content, instructional practices, or peer groups that 
meet them where they are. The primary goals of acceleration are, according to the National 
Association of Gifted Children (NAGC) in its 2004 position paper, to “adjust the pace of 
instruction to the students’ capability in order to develop a sound work ethic, to provide an 
appropriate level of challenge in order to avoid the boredom from repetitious learning, and to 
reduce the time period necessary for students to complete traditional schooling” (Children, 
2004). The group goes to on to endorse acceleration “as one important avenue to address the 
needs of gifted learners” (NAGC, 2004). While numbers or percentages of students engaged in 
content-based acceleration are hard to come by, roughly three-quarters of district- and school- 
based staff in select geographic locales have reported using this form of acceleration (Guilbault, 
2009; Kanevsky, 2011). 


Early research on the effectiveness of acceleration focused on the extent to which 
accelerated students were harmed, either academically or socially, in the process of completing 
advanced coursework or working alongside older classmates (Neihart, 2007). However, a 
wealth of descriptive research contained in the second volume of A Nation Deceived suggests 
that accelerated students are in fact largely insulated from any harm that may come as a result 
of acceleration (Colangelo et al., 2004; Kulik & Kulik, 1984). 


Less well known, however, is how well accelerated students perform compared with 
their non-accelerated peers. A number of meta-analytic studies conclude that they do 
outperform their counterparts, but much of the research cited in these reviews consists of 
descriptive analyses, case studies, small-sample evaluations, and studies lacking methodological 
rigor. For example, the first quantitative review of subject-acceleration found overwhelmingly 
positive effects for those students who were accelerated. In particular, accelerated students 
who were compared with similarly-aged peers who were not accelerated outperformed them 
by 0.88 standard deviations (SD) on a variety of academic measures (Kulik & Kulik, 1984). 
However, the control groups in these studies were not derived from random assignment but 
rather from crude and potentially biased matching methods (e.g., matching students on similar 
IQ scores). A more recent review found an effect size of 0.18 SD across 28 studies on a range of 
acceleration strategies and an effect of 0.06 SD for 11 studies on content-based acceleration, 
but neither of these impacts was statistically significant (Steenbergen-Hu & Moon, 2011). 
Moreover, only six of the 28 studies in the analysis used experimental or quasi-experimental 
design, limiting our ability to conclude that acceleration—and not some other factor—caused 
these outcomes to occur. This study contributes to the small number of quasi-experimental 
explorations of subject-based acceleration 


Single Subject Acceleration in WCPSS 


WCPSS adopted its broad acceleration policy in fall 2010 and revised it twice since. 
Board Policy 5532? includes a range of acceleration options—including subject-based—but does 
not specify the criteria students must meet in order to qualify. The criteria for SSA are 
developed and maintained by the district’s implementation team. While qualifying criteria have 
changed since the first year of implementation, the basic pathway from nomination to 
acceleration has remained consistent. In order to be considered for SSA, parents, teachers or 
students themselves must submit a nomination during the window (typically April and early 
May) for acceleration in either mathematics or reading, but not both subjects. Each school’s 
School Based Committee for Gifted Education (SBCGE) then reviews nominations for 
completeness in the spring and schedules nominees for qualifying testing. Prior to testing, SSA 
implementation team members conduct evening information sessions for parents at select 
elementary and middle schools. Testing itself occurs between the 161st and 170th day of each 
school’s calendar, which translates into Track 1 and Modified schools testing in early-to-mid 
May, Traditional schools testing in mid-to-late May, and Tracks 2-4 testing during the first two 
weeks of June. 


While content-based acceleration was not new to the district in 2013-14, the use of 
specific criteria in determining whether a student could take accelerated content was. The 
catalyst for establishing specific criteria was the district’s desire to create an equitable process 
through which any student, regardless of AIG classification, could benefit from acceleration. In 
the past, students were considered candidates for SSA if they consistently performed above 
grade-level standards in mathematics or reading, were independent learners who thrived in the 
face of academic challenges, and were socially and emotionally mature enough to interact with 
older classmates. While these characteristics were widely viewed as necessary conditions for 
acceleration, they were not sufficient. In addition to subjective criteria, WCPSS students could 
qualify in spring 2013 to take accelerated content during the 2013-14 school year if they met 
the following three criteria: 


e Completion of a student portfolio requiring 1-2 years of content mastery based on the 
Grade Level Portfolio Component Checklist; 

e Score greater than or equal to 95% on the Cognitive Abilities Test (CogAT); and 

e Score greater than or equal to 98% on the lowa Test of Basic Skills (ITBS) or one grade 
level above the current grade placement. 


Around the time of the first SSA qualifying administration, district leadership decided to 


remove CogAT and ITBS testing in order to reduce the number of qualifying tests, streamline 
the process, and more fully comply with a then forthcoming State Board of Education policy, 


3 Visit https://webarchive.wcpss.net/policy.html for more detail about Board Policy 5532. 


which ultimately permitted content-based acceleration within the Credit by Demonstrated 
Mastery (CDM) framework (NCSBE & NCDPI, 2013). 


Current Qualifying Criteria 

In order to measure the impact of SSA mathematics using a single qualifying criterion, 
we omitted the 2013-14 school year and focused on first-time nominees who qualified in the 
spring prior to the 2014-15 and 2015-16 school years. While students enrolled in grades K-7 
could be nominated for SSA, this analysis omits kindergarten students hoping to skip grade 1 
mathematics because central office staff only had qualifying rosters for these students and not 
qualifying scores,* which are necessary for this analysis. Table 3 summarizes the qualifying 
criteria for students wishing to skip content in grades 2-5, 6 PLUS, or 7 PLUS.° Notably, 
elementary students who qualify to take a middle school mathematics course must remain in 
their assigned elementary school and typically take the course online. 


Table 3 
Qualifying Assessment Criteria for SSA Mathematics 
Grade/Course Assessment Used to Qualifying Criterion Standards for the 
Requesting to Skip Qualify Grade/Course 
Comprehensive above Grageriever Common 
2-5 siadeleveliaesecement = 80% correct Core State Standards 
(CCSS) 
6 PLUS Comprehensive above Sepanek Mathematics 6 PLUS 
grade level assessment CCSS 
7 PLUS Comprehensive above Lene ane Mathematics 7 PLUS 
grade level assessment CCSS 


Source: AIG department. The assessment developer for the qualifying assessment is TE21, Inc. 


METHODS 


To determine the impact of single subject acceleration (SSA) on summative mathematics 
outcomes, we used administrative data from WCPSS, which includes demographic and special 
program indicators, and End-of-Grade (EOG) mathematics test scores. The EOG mathematics 
test is administered to all students in grades three through eight in the spring. Because 
accelerated students receive mathematics content for a higher grade level, the most logical 
outcome of interest would be the EOG test covering the accelerated content. However, various 
state testing policies require students to take the grade-level EOG test aligned to their official 


4 Kindergarten nominees hoping to skip grade 1 mathematics are administered the First Grade Mathematics 
Summative Assessment, developed by the North Carolina Department of Public Instruction. The 2016-17 version of 
this assessment includes 14 separate tasks on which students can earn a Level 1-3. To qualify for SSA, nominees 
must score a Level 3 on each task. 

> Math 6 Plus is a compacted course comprised of all of the Math 6 standards and a portion of the Math 7 
standards. Math 7 Plus is a compacted course comprised of a portion of standards from Math 7 and a portion of 
standards from Math 8. Source: WCPSS Middle School Planning Guide. 


grade-level classification.° Accelerated students do return to their grade-level mathematics 
class for roughly two weeks to prepare for the grade-level EOG test, but that is their only 
exposure to peers or grade-level content—for which they already demonstrated mastery— 
during the entire school year. 


The variable that determines whether students qualify for acceleration is the qualifying 
assessment score for the accelerated content. For example, a 3rd grade student in spring 2015 
wishing to skip 4th grade mathematics would need to score at or above 80% on this assessment 
in order to take 5th grade mathematics upon entering 4th grade in the 2015-16 school year. 
The qualifying assessment score is known as the “assignment” variable’ that places students in 
either their originally scheduled class (if they score below 80%) or to SSA mathematics (if they 
score 80% or higher). 


In order to measure the impact of being accelerated on math achievement, we focus on 
the sample of students close to the 80% qualifying cutoff because students around this score 
are expected to differ only on their score and not in other substantial ways. In other words, 
students who scored a 75% are, in theory, quite similar to students who scored an 85%—yet 
only the latter group qualifies for SSA. A measurement technique known as regression 
discontinuity design (RDD) allows us to measure impacts by taking the average difference in 
achievement between accelerated and non-accelerated students around the 80% cutoff. The 
appendix includes technical details about RDD, but graphics presented in the next section 
provide a method for interpreting RDD visually. 


RESULTS 


Descriptive Data 

Students in our sample fell into three categories: nominated, qualified, and accelerated. 
Students were nominated by themselves, parents/guardians or teachers and subsequently 
were given a chance to qualify for SSA by taking the qualifying assessment for the next grade 
level. Nominated students who scored 80% or above on that test qualified for SSA. Qualifying 
students who ultimately enrolled in the accelerated class were labeled as accelerated. Table 4 
shows the proportion of nominees who qualified and the proportion of qualifiers who 
ultimately accelerated. Among nominees, the rate of qualification in the two years under study 
has ranged from 29% to 31%, while the rate of acceleration has ranged from 17% to 21%. Out 
of the entire pool of nominees over the two-year period, the combined qualification and 
acceleration rates were 30% and 22%, respectively. 


Table 5 shows that a relatively large proportion of male, White, Asian, and AIG students 
were nominated for SSA, qualified and ultimately accelerated. From nomination to 


§ See http://www.ncpublicschools.org/docs/accountability/1617reqtstalt.pdf for information about the various 
state and federal requirements that govern EOG test administration. 
7 This variable is also known as the “forcing” variable or “running” variable (Lee & Lemieuxa, 2010) 


acceleration, the relative proportion of male students to female students consistently increased 
and remained steady for Asian, White, and academically gifted students. Compared with district 
averages for students in those grades eligible for SSA, a much smaller percentage of female, 
Black, Hispanic/Latino, Limited English Proficient (LEP) students and students with disabilities 
(SWD) were nominated for SSA mathematics, qualified and ultimately accelerated. Prior 
achievement in mathematics for nominated students was 1.14 standard deviations higher than 
the district average and roughly 1.5 standard deviations higher for qualified and accelerated 
students. For context, the national White-Black achievement gap is roughly 0.5-1.0 standard 
deviations.® 


Table 4 
SSA Mathematics Nominees, Qualifiers and Accelerators, 2014-15 and 2015-16 
2015 | 2016 Pooled 
Nom. | Qual. — Acc. | Nom. | Qual. | Acc. Nom. — Qual. | Acc. 
N 1,711 500 307. «1,326 «= 414. is 357) 3,087's—«i914ts«iG 
% 100% 29.2% 17.9% 100% 31.2% 26.9% 100% 30.1% 21.9% 
Source: WCPSS administrative data ) ) 
Table 5 
SSA Math Nominees, Qualifiers and Accelerators, by Subgroup, 2014-15 and 2015-16 
WCPSS Nominated Qualified Accelerated 
N % N % N % N % 
Female 72,773 48.9 1,165 38.4 301 32.9 208 31.3 
Male 76,204 51.3 1,872 61.6 613 67.1 456 68.7 
Asian 11,877 8.0 1,149 37.8 405 44.3 277 41.7 
Black 34,005 22.8 146 4.8 19 2.0 15 2.3 
Hispanic/Latino 26,331 17.7 100 3.3 20 2.2 17 2.6 
White 70,430 47.3 1,519 50.0 431 47.2 326 49.0 
Multiracial 5,/67 3.9 114 3.8 35 3.8 27 4.1 
AIG 26,760 18.0 1,293 42.6 464 50.8 333 50.1 
LEP 12,426 8.3 54 1.8 12 1.3 11 1.7 
SWD 19,533 13.1 74 2.4 20 2.2 11 1.7 
Prior EOG math (SD) 0.00 _— 1.14 _— 1.51 _— 1.48 _— 


Note: AIG: Academically and Intellectually Gifted; LEP: Limited English Proficient; SWD: Students with Disabilities; 
SD stands for standard deviation units. Source: WCPSS administrative data. 


8 “Racial and Ethnic Achievement Gaps,” The Educational Opportunity Monitoring Project, Stanford Center for 
Education Policy Analysis. See http://cepa.stanford.edu/educational-opportunity-monitoring- 
project/achievement-gaps/race. 
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One important feature of RDD analysis is the nature of the distribution of qualifying 
scores around the cutoff, since we are ultimately comparing students who fall just below and 
just above the 80% threshold. Figure 2 shows the combined distribution of qualifying scores for 
spring 2014 and spring 2015 (the distributions are similar for each year). The distribution 
exhibits a negative, or left-skewed, pattern, with a mean score of 67.3% (standard deviation 
(SD): 16.7; range: 6-100). 


Figure 2 
Distribution of SSA Qualifying Scores, Spring 2014 and Spring 2015 
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Note: Sample includes 3,037 SSA nominees who took the TE21 CASE qualifying test. 
Source: TE21, Inc. assessment data. 


A critical feature of valid RDD analysis is the absence of any deliberate manipulation 
around the cutoff score. This could be exhibited by a spike in the frequency of scores at the 80% 
cutoff. While an ideal cutoff score is unknown to the test-taker, this was not the case with 
qualifying assessment scores. The AIG Department actively marketed SSA to various 
stakeholders throughout the district and the cutoff score was well known. There was nothing 
wrong with this per se—it has merely been a transparent feature of the initiative. Moreover, 
students may have had access to released items from a previous version of the qualifying test, 
thus making it possible that they could use that information to try and improve their score in 
advance of their qualifying test. 
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To determine whether such manipulation of scores was evident, we restricted the visual 
distribution of qualifying scores around the cutoff. Examining the sample of qualifying scores 
within a range of 15 points around the cutoff in Figure 3, there does not appear to be 
manipulation around the cutoff, which would be signaled by a spike at 80%. If anything, a spike 
occurs around 70%, sufficiently below the cutoff. The apparent test score gaps to the left of the 
cutoff appear because no students happened to earn these scores. 


Figure 3 
Distribution of CASE Qualifying Scores, +/- 15 Points around the Cutoff 
Spring 2014 and Spring 2015 
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CASE Test Score (Spring 2014-2015) 


Note Sample includes 1,735 SSA nominees with CASE scores between 65% and 95%, inclusive. 
Source: TE21, Inc. assessment data. 


After examining the distribution of qualifying scores and confirming that manipulation 
around the cutoff was unlikely, we explored the degree to which SSA was actually 
implemented. Table 6 shows that a combined 3,037 students were nominated in spring 2014 
and spring 2015. From the entire pool of nominees—those who took the qualifying test—914 
qualified for SSA mathematics and 664 were ultimately accelerated. The table shows the 
numbers and percentages of compliers and non-compliers in our sample of nominees. Among 
those who were nominated but did not qualify, 74 students still managed to accelerate for 
reasons that are not known. And among those who qualified for acceleration, 324 students 
(35% of those who qualified) ultimately did not accelerate. Nearly two-thirds among these 
occurred in 2014-15, suggesting that it was less of an issue in the 2015-16 school year. Still, in 
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both years, the greatest concentration of qualified non-accelerators occurred among students 
in grades 2 and 5. Because the sample has, for whatever reason, students who either 
accelerated without qualifying or failed to accelerate despite qualifying, we utilized a so-called 
“fuzzy” RDD strategy. The term fuzzy contrasts with “sharp,” which would indicate perfect 
compliance in which 100% of qualifiers accelerated and 0% of non-qualifiers accelerated. 


Table 6 
SSA Compliance among Nominees 
Did not accelerate Accelerated | Row Totals 

Did Not Qualify 2,049 74 2,123 

(96.5%) (3.5%) (100%) 
Qualified 324 590 / 914 

(35.4%) (64.6%) | (100%) 
Column Totals 2,373 664 3,037 

(78.1%) (21.9%) (100%) 


Source: WCPSS administrative data 


Figure 4 graphically demonstrates this fuzzy nature of SSA take-up. If take-up were 
“sharp,” 0% of students below the 80% qualifying cutoff score would have accelerated and 
100% at or above the 80% cutoff score would have accelerated. The scatter on the left of the 
cutoff shows that some students accelerated who did not qualify and the scatter on the right of 
the cutoff shows that some students who qualified did not accelerate. Since nearly two-thirds 
of qualifiers ultimately accelerated, we proceed to measure the impact of participating in SSA 
on EOG test scores. 
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Figure 4 
Percent Accelerated, +/- 15 Points around the Cutoff 
SSA Qualifiers, Spring 2014 & 2015 
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CASE Test Score (Spring 2014-2015) 


Note: Sample includes 1,735 SSA nominees with CASE scores between 65% and 95%, inclusive. 
Source: TE21, Inc. assessment data. 


Impacts 

To measure the impact of SSA on student achievement, we use the grade-level (i.e., 
non-accelerated) EOG test as our outcome. Recall that while students who are accelerated in 
mathematics participate in an advanced course, they are—by law—administered the EOG that 
corresponds with their actual grade level, not their advanced course. If SSA impacted EOG 
performance, it would suggest that accelerated students outperformed their grade-level 
counterparts who did not accelerate but who are actually enrolled in the course with tested 
content. If SSA did not impact grade-level EOG performance, we would say that accelerated 
students perform just as well as their grade-level counterparts who are enrolled in the tested 
course. In such a case, we would conclude that the accelerated students’ presence in a non- 
tested course did not harm their mathematics achievement and at the same time provided 
them with the opportunity to take advanced content. 


Before measuring the precise impact of SSA on EOG performance, we first look for a 
discontinuity in achievement around the cutoff. The impact of SSA is expressed in standard 
deviation units (SD). SD units permit us to combine the outcome across multiple years and 
compare impacts, if any exist, to education interventions in different settings. Figure 5 shows 
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the relationship between the SSA qualifying score and performance on the EOG test during the 
accelerated year. If an impact existed, we would expect to see a break, or discontinuity, around 
the 80% cutoff score. An increase at the dotted cutoff line to the right of 80% would suggest 
that accelerated students may have outperformed their non-accelerated counterparts. While 
the visual across all qualifying scores in Figure 5 makes it difficult to see a discontinuity, the 
condensed chart in Figure 6 more clearly shows the discontinuity around the cutoff. Here, it 
appears that accelerated students performed nearly 0.1 SD higher than their non-accelerated 
counterparts at the cutoff (0.5 SD -0.4 SD = 0.1 SD). 


Figure 5 
CASE Test Score and Summative EOG Mathematics Achievement 


EOG Math Score, 2015-2016 (Standardized) 
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Note: Sample includes 2,501 SSA nominees with CASE and EOG test scores. Sources: WCPSS and TE21, Inc. 
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Figure 6 
CASE Qualifying Score and Summative EOG Mathematics Achievement 
+/- 15 Points around Qualifying Cutoff 


EOG Math Score, 2015-2016 (Standardized) 


CASE Test Score, Spring 2014-2015 


Note: Sample includes 1,292 SSA nominees with CASE and EOG test scores. 
Sources: WCPSS and TE21, Inc. assessment data. 


To determine whether this visual inspection holds up after controlling for a host of 
variables, we use a graphical display called a coefficient plot to show impacts across a range of 
statistical models and “bandwidths” around the qualifying cutoff score of 80%. Varying the 
bandwidth around the cutoff—i.e., + 20 points down to + 5 points—gives us more confidence in 
our results. The coefficient plots in Figure 7 include six different statistical models.’ The use of 
multiple models gives us additional confidence that the results we see are reliable. In the plot, 
the impact is illustrated with a marker that sits above or below the dotted line marked with a 
zero. If the marker is above the dotted line, the impact is positive and if it’s below the line, the 
impact is negative. If the vertical lines radiating from the marker—known as confidence 


° The six models are (1) a bivariate regression model that measures the impact of SSA only on the outcome, 
omitting controls, (2) controlling for demographics, (3) controlling for prior achievement, (4) controlling for 
variation by cohort (i.e., students who accelerated in 2014-15 may vary in some way from students who 
accelerated in 2015-16), (5) controlling for school-level variation, since some schools may have implemented SSA 
differently from others and (6) controlling for grade-level variation (i.e., acceleration may vary widely depending 
on the grade-level content that is skipped). 
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intervals (Cl)—touch the dotted line marked zero, then the impact was not statistically different 
from zero. 


Figure 7 shows four different coefficient plots, each with six impact models. The first 
major finding is that SSA did not statistically impact grade-level EOG test performance across 24 
different models since no single marker and its corresponding Cl clears the zero line. The 
second major finding is that accelerated students relatively farther above the cutoff (i.e., 20 or 
15 points) performed similarly to their non-accelerated counterparts. Third, accelerated 
students closer to the cutoff—10 points—performed slightly better than their non-accelerated 
counterparts, but this impact was not statistically different from zero. Finally, closest to the 
cutoff—5 points—accelerated students appear to have performed slightly worse than their 
counterparts, but again, this result was not statistically different from zero. 
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Figure 7 
Impacts of SSA on EOG Mathematics Achievement at Various Ranges around Cutoff 
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Note: In the plot above, the impact is illustrated with a marker that sits above or below the dotted line marked with a zero. If the marker is above the dotted line, the impact is positive and if it’s below 
the line, the impact is negative. If the vertical lines radiating from the marker—known as confidence intervals (Cl)—touch the dotted line marked zero, then the impact was not statistically different 
from zero. The sample size becomes smaller as the range around the cutoff shrinks. From the largest range to the smallest (+/- 20 points to +/- 5 points) the sample size was reduced from 1,567 
students to 566 students. In the bottom right panel, the y-axis is double the range of the y-axis in the previous three panels. 
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Conclusions 


Single-subject acceleration (SSA) is a form of subject-level acceleration designed to 
provide high-achieving students with advanced coursetaking opportunities. A large body of 
research consisting of case studies and correlational findings suggests that at best, SSA 
contributes to increases in student achievement and, at worst, does no harm. This study is 
among the first to measure the causal impact of SSA on mathematics achievement and in doing 
so, concludes that students who enroll in accelerated mathematics through SSA perform no 
differently on grade-level mathematics assessments than their non-accelerated counterparts. 


This “do-no-harm” finding does not suggest that SSA is an ineffective intervention for 
high-achieving students. This is because accelerated students are not assessed for their mastery 
of accelerated content. Rather, they are assessed on their grade-level content along with their 
non-accelerated peers. By state policy, students must take the grade-level EOG and are not 
allowed to sit for the content-level test with peers from their accelerated course. The added 
value of SSA may not be marginally higher grade-level test scores for already high-achieving 
students, but the exposure to advanced mathematics could conceivably have an additive effect 
in subsequent years. Thus, as long as accelerated students are not underperforming their non- 
accelerated counterparts on the content they are skipping, the opportunity for accelerated 
mathematics study may be sufficient to justify the initiative. 


Recommendations 


Expand the visibility of SSA in order to inform more diverse populations. Compared with 
district averages, a disproportionately low percentage of female, Black and Hispanic/Latino 
students were nominated for SSA, qualified, and ultimately accelerated. To address this gap, 
the SSA implementation team should broaden its efforts to promote SSA to schools, 
communities and other stakeholders that may not have sufficient information required to 
pursue this opportunity. It should be noted that this is a necessary, though not sufficient, 
condition for equity in nomination, qualification or acceleration among underrepresented 
groups. That is, even if every student in the district were nominated for SSA and administered 
the qualifying test, achievement gaps would still remain that mirror existing gaps across a range 
of different outcomes. 


Identify potential causes of non-acceleration among qualifiers. Roughly one-third of students 
who qualified for SSA in mathematics did not ultimately enroll in the accelerated course. We do 
not know exactly why qualified students did not accelerate, but we do know that these 
students were concentrated in the 2014-15 school year, in grades 2 and 5 in both years under 
study, and in only a handful of schools. Evaluation and implementation team members should 
closely explore the reasons for non-compliance in order to increase the numbers of qualified 
students eligible to accelerate to ensure that SSA is available to all students who qualify. 
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Explore options for assessing content-level mastery. Due to state and federal regulations, 
students must take the summative assessment only for the grade level in which they are 
enrolled. This means a 3rd grade student who is accelerated to take 4th grade mathematics 
cannot take the grade 4 End-of-Grade test. District staff should consider strategies for assessing 
content-level mastery for accelerated students in ways that do not burden these students with 
too much additional testing or run afoul of state and federal assessment regulations. This would 
give us a clearer understanding of the potential benefits of SSA. 


Maintain the 80% qualifying CASE score for SSA mathematics. Following two years of SSA 
implementation, the qualifying cutoff of 80% in mathematics appears reasonable. A lower 
qualifying score would allow more students to accelerate and a higher score would allow fewer 
to accelerate. Since SSA appears to function as a do-no-harm intervention whereby accelerated 
students perform similarly to their non-accelerated counterparts and receive the benefit of 
exposure to advanced mathematics content, we believe the initiative is largely working as 
intended from the standpoint of student achievement. 
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APPENDIX 


Regression discontinuity design (RDD) allows us to compare the outcomes for students 
who were just below and above the qualifying cutoff score (Imbens & Lemieux, 2008). Those 
below the score were nominated and did not qualify while those above the qualifying score were 
nominated and did qualify to participate in SSA. We expect a comparison of students just below 
and above the qualifying cutoff score to yield unbiased estimates of SSA’s impact on summative 
academic outcomes. Our outcome variable is the summative EOG test. We use as controls 
individual level indicators for sex, race, gifted status, LEP status, disability status, and 
corresponding school-level rates. 


To estimate the impact of qualifying for SSA on student outcomes, we first estimate intent- 
to-treat (ITT) impacts using the following two-level random effects specification: 


Yi; = Bo + B, Qualified;, + B,CASE;, + B3(Qualified * CASE), + Zis + Vis + Hi 


Where Y represents an outcome interest for student / in school s who qualified in spring 2014 to 
enroll in an accelerated mathematics course during the 2014-15 school year. The variable 
Qualified indicates that the student met the 80% eligibility threshold on the spring 2014 CASE test. 
This coefficient indicates the difference in acceleration rates between students just below and just 
above the qualifying cutoff score. The variable CASE represents the continuous qualifying values of 
the test, which is re-centered around the qualifying threshold of 80. The coefficient for this 
variable represents the slope of the relationship between the probability of acceleration and CASE 
scores to the left of the 80% qualifying score. The interaction of Qualified and CASE allows the 
slope to vary to the right of the 80% qualifying threshold. 


Because some students who qualified for SSA did not ultimately accelerate, we employ 
fuzzy RDD by utilizing a two-stage least squares model to generate treatment-on-treated (TOT) 
estimates (Imbens & Lemieux, 2008). In the first stage, we use the qualifying threshold as source 
of arguably exogenous variation: 


Accelerated;, 
=X y+ X, Qualified;, + x, CASE;, + %3 (Qualified * CASE);, + Zis + Eis 
+ A; 


Where Accelerated indicates that student i in school s enrolled in an accelerated mathematics 
course in 2014-15 after qualifying in spring 2014. The indicator Qualified indicates that a student 
met the 80% eligibility threshold on the spring 2014 CASE test. The continuous variable TE21 is the 
range of scores within the +15 percentage point bandwidth (or other, reduced bandwidths) on 
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either side of the 80% cutoff, re-centered around the cutoff. The predicted values from the first 
stage, above, are used to estimate the second-stage: 


Yi, = Bo + B,Accelerated;, + B2,CASE;, + B3(Qualified * CASE);, + Zis + Vis + Hi 


Where Y represents an outcome interest for student jin school s, such as summative EOG test 
scores in spring 2015 and spring 2015. Thus, 6; is our coefficient of interest and represents the 
impact of SSA on later mathematics outcomes for students who actually enrolled in an accelerated 
mathematics course. 
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